Comparing Learning Classifier Systems for Continuous-Valued Online Environments

We investigate Learning Classifier Systems for online environments that consist of real-valued states and which require every action made by the agent to count towards its performance. Two Learning Classifier System architectures are considered, ZCS and XCS. We use an interval representation with these Learning Classifier Systems for the rule conditions together with roulette wheel action selection. As real-world environments are rarely deterministic, we investigate the performance of these two Learning Classifier System architectures on a set of artificial environments with stochastic reward functions. We briefly review related work and relate this to the experiments performed in this paper. Although XCS clearly delivers superior performance in deterministic environments, we find that the simple ZCS architecture is robust and able to equal or exceed the performance of XCS in stochastic environments, especially those with more demanding characteristics.